Detection of attachment problem of apparatus being worn by user

ABSTRACT

Provided is to prevent a false determination due to an attachment condition of an apparatus that transmits and receives an acoustic signal, and perform accurate personal authentication. A personal authentication device includes: a personal authentication means that authenticates an individual by using first information at least including an acoustic characteristic calculated from an acoustic signal propagating through the head of the user, which is detected by an apparatus being attached on a head of a user for transmitting and receiving the acoustic signal, and a feature amount extracted from the acoustic characteristic; an attachment trouble rule storage means that stores an attachment trouble rule for detecting an attachment trouble with the apparatus; and an attachment trouble detection means that detects a trouble with an attachment state of the apparatus when the first information satisfies the attachment trouble rule.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 18/070,875 filed on Nov. 29, 2022, which is acontinuation application of U.S. patent application Ser. No. 16/325,849filed on Feb. 15, 2019, which issued as U.S. Pat. No. 11,537,695, whichis a National Stage Entry of international application No.PCT/JP2017/028531 filed on Aug. 7, 2017, which claims the benefit ofpriority from Japanese Patent Application No. 2016-161413 filed on Aug.19, 2016, the disclosures of all of which are incorporated in theirentirety by reference herein.

TECHNICAL FIELD

The present invention relates to a personal authentication system, apersonal authentication device, a personal authentication method, apersonal authentication program.

BACKGROUND ART

NPL 1 in the technical field discloses a technique for measuring anacoustic characteristic of an ear canal of a user by using an earphonewith a built-in microphone, and performing personal authentication byusing a difference between individuals in the measured acousticcharacteristic of the ear canal.

CITATION LIST Non Patent Literature

-   [NPL 1] Takayuki ARAKAWA et al., “Accurate Person Authentication    Using Ear Canal Acoustic Features”, 2-P-30, Paper at 2016 Spring    Meeting of the Acoustical Society of Japan

SUMMARY OF INVENTION Technical Problem

The techniques described in the above documents do not mentionpossibility of misjudgment and solution thereof, caused by a malfunctionin the installation of equipment that transmits and receives soundsignals such as headphones and earphones.

An object of the present invention is to provide a technique for solvingthe above-described problem.

Solution to Problem

To achieve the above object, a personal authentication device accordingto the present invention includes:

-   -   a personal authentication means that authenticates an individual        by using first information including at least one of        -   an acoustic characteristic calculated from an acoustic            signal propagating through the head of the user, which is            detected by an apparatus being attached on a head of a user            for transmitting and receiving the acoustic signal, and        -   a feature amount extracted from the acoustic characteristic;    -   an attachment trouble rule storage means that stores an        attachment trouble rule for detecting an attachment trouble with        the apparatus; and    -   an attachment trouble detection means that detects a trouble        with an attachment state of the apparatus when the first        information satisfies the attachment trouble rule.

To achieve the above object, a personal authentication method accordingto the present invention includes:

-   -   authenticating an individual by using first information        including at least one of        -   an acoustic characteristic calculated from an acoustic            signal propagating through the head of the user, which is            detected by an apparatus being attached on a head of a user            for transmitting and receiving the acoustic signal, and        -   a feature amount extracted from the acoustic characteristic;            and    -   detecting a trouble with an attachment state of the apparatus        when the first information satisfies an attachment trouble rule        for detecting an attachment trouble with the apparatus.

To achieve the above object, a personal authentication program accordingto the present invention causes a computer to execute:

-   -   processing of authenticating an individual by using first        information including at least one of        -   an acoustic characteristic calculated from an acoustic            signal propagating through the head of the user, which is            detected by an apparatus being attached on a head of a user            for transmitting and receiving the acoustic signal, and        -   a feature amount extracted from the acoustic characteristic;            and    -   processing of detecting a trouble with an attachment state of        the apparatus when the first information satisfies an attachment        trouble rule for detecting an attachment trouble with the        apparatus.

To achieve the above object, a personal authentication system accordingto the present invention includes:

-   -   an apparatus that transmits and receives an acoustic signal at a        head of a user;    -   a personal authentication means that authenticates an individual        by using first information which is including at least one of        -   an acoustic characteristic calculated from an acoustic            signal that is detected by the apparatus being attached and            that propagates through the head of the user, and        -   a feature amount extracted from the acoustic characteristic;    -   an attachment trouble rule storage means that stores an        attachment trouble rule for detecting an attachment trouble with        the apparatus; and    -   an attachment trouble detection means that detects a trouble        with an attachment state of the apparatus when the first        information satisfies the attachment trouble rule.

Advantageous Effects of Invention

According to the present invention, it is possible to prevent erroneousdetermination due to the wearing state of a device fortransmitting/receiving an acoustic signal, and to perform accuratepersonal authentication.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a personalauthentication device according to a first example embodiment of thepresent invention.

FIG. 2 is a diagram illustrating an overview of a personalauthentication system including a personal authentication deviceaccording to a second example embodiment of the present invention.

FIG. 3 is a block diagram illustrating a functional configuration of thepersonal authentication device according to the second exampleembodiment of the present invention.

FIG. 4A is a diagram illustrating an acoustic characteristic used forpersonal authentication according to the second example embodiment ofthe present invention.

FIG. 4B is a diagram illustrating an acoustic characteristic used forpersonal authentication according to the second example embodiment ofthe present invention.

FIG. 5A is a diagram illustrating a configuration of an attachmenttrouble rule storage unit according to the second example embodiment ofthe present invention.

FIG. 5B is a diagram illustrating a configuration of an attachmenttrouble detection table according to the second example embodiment ofthe present invention.

FIG. 6 is a block diagram illustrating a hardware configuration of thepersonal authentication device according to the second exampleembodiment of the present invention.

FIG. 7 is a flowchart illustrating a processing procedure of thepersonal authentication device according to the second exampleembodiment of the present invention.

FIG. 8 is a flowchart illustrating a procedure for attachment troubledetermination processing according to the second example embodiment ofthe present invention.

FIG. 9 is a block diagram illustrating a functional configuration of apersonal authentication device according to a third example embodimentof the present invention.

FIG. 10A is a diagram illustrating a configuration of an attachmenttrouble rule storage unit according to the third example embodiment ofthe present invention.

FIG. 10B is a diagram illustrating a configuration of an acousticcharacteristic storage unit according to the third example embodiment ofthe present invention.

FIG. 10C is a diagram illustrating a configuration of an attachmenttrouble detection table according to the third example embodiment of thepresent invention.

FIG. 11 is a flowchart illustrating a procedure for attachment troubledetermination processing according to the third example embodiment ofthe present invention.

FIG. 12 is a block diagram illustrating a functional configuration of apersonal authentication device according to a fourth example embodimentof the present invention.

FIG. 13 is a diagram illustrating a feature amount used for personalauthentication according to the fourth example embodiment of the presentinvention.

FIG. 14A is a diagram illustrating a configuration of an attachmenttrouble rule storage unit according to the fourth example embodiment ofthe present invention.

FIG. 14B is a diagram illustrating a configuration of an attachmenttrouble detection table according to the fourth example embodiment ofthe present invention.

FIG. 15 is a flowchart illustrating a processing procedure of thepersonal authentication device according to the fourth exampleembodiment of the present invention.

FIG. 16 is a block diagram illustrating a functional configuration of apersonal authentication device according to a fifth example embodimentof the present invention.

EXAMPLE EMBODIMENT

Hereinafter, example embodiments of the present invention areillustratively described in detail with reference to drawings. However,a structural component described in the example embodiments below ismerely an exemplification, and a technical scope of the presentinvention is not intended to be limited to the structural component.

First Example Embodiment

Personal authentication device 100 as a first example embodiment of thepresent invention is described by using FIG. 1 . Personal authenticationdevice 100 is a device for authenticating an individual.

As illustrated in FIG. 1 , personal authentication device 100 includespersonal authentication unit 101, attachment trouble rule storage unit102, and attachment trouble detection unit 103. Personal authenticationunit 101 authenticates an individual by using, as first information, atleast any of an acoustic characteristic calculated from an acousticsignal propagating through a head of a user wearing an apparatus thattransmits and receives the acoustic signal and a feature amountextracted from the acoustic characteristic. Attachment trouble rulestorage unit 102 stores an attachment trouble rule for detecting anattachment trouble with the apparatus. Attachment trouble detection unit103 detects a trouble with an attachment state of the apparatus when thefirst information satisfies the attachment trouble rule.

According to the present example embodiment, when an acousticcharacteristic acquired from an acoustic signal propagating through ahead of a user and a feature amount satisfy an attachment trouble rulefor detecting an attachment trouble with an apparatus, a trouble isdetected in an attachment state of the apparatus. In this way, a falsedetermination due to an attachment condition of the apparatus thattransmits and receives an acoustic signal can be prevented, and accuratepersonal authentication can be performed.

Second Example Embodiment

Next, a personal authentication device according to a second exampleembodiment of the present invention is described. The personalauthentication device according to the present example embodiment storesan attachment trouble rule for detecting an attachment trouble with awearable apparatus, and detects a trouble with an attachment state ofthe apparatus when an acoustic characteristic acquired from an acousticsignal by a personal authentication unit satisfies the attachmenttrouble rule for detecting an attachment trouble with the apparatus. Inthe present example embodiment, the attachment trouble rule is a rulethat determines an attachment trouble, based on a comparison between anacoustic characteristic and a predetermined reference. Then, the troublewith the attachment state of the apparatus is notified.

(Acoustic Characteristic)

Note that, for example, an impulse response, a transfer functionacquired by performing a Fourier transform or a Laplace transform on animpulse response are conceivable as an acoustic characteristic. Theacoustic characteristic includes, for example, information about how anacoustic signal is reflected and/or declines in a living body. When anearphone and a microphone are attached at an ear canal opening, and anacoustic characteristic reflected in the ear canal is measured, an earcanal impulse response and an ear canal transfer function may be used asan acoustic characteristic.

<<Overview of Personal Authentication System>>

FIG. 2 is a diagram illustrating an overview of personal authenticationsystem 200 including personal authentication device 210 according to thepresent example embodiment.

Personal authentication system 200 illustrated in FIG. 2 includespersonal authentication device 210 as a personal computer (PC), attachedapparatus 220 including an earphone and a microphone, sound processor230, and microphone amplifier 240. Note that, reference sign 250indicates a user (subject) to be recognized.

Note that, it is desirable that a microphone and an earphone areintegrally formed in such a way as not to change a relative positionalrelationship thereof. However, this is non-essential when a relativepositional relationship between a microphone and an earphone does notgreatly change. Further, a microphone-integrated earphone that isinserted into an ear canal entrance is taken as attached apparatus 220herein, but a microphone set in a headphone being a type of covering anauricle may be taken as an example of achieving both a microphone and anearphone. Further, a microphone set in a portion of a telephone receivermay be taken as another example of achieving both a microphone and anearphone. In addition, observing an acoustic signal transmitted from anearphone attached at a left ear canal entrance by a microphone attachedat an ear canal entrance of a right ear and vice versa, are alsoconceivable.

In FIG. 2 , personal authentication device 210 has detected anattachment trouble with attached apparatus 220, and thus voice of“re-attach earphone” is output from a speaker, or “earphone attachementerror!” is displayed on a screen. Note that, an attachment trouble maybe notified by an alarm from attached apparatus 220 instead of personalauthentication device 210.

<<Functional Configuration of Personal Authentication Device>>

FIG. 3 is a block diagram illustrating a functional configuration ofpersonal authentication device 210 according to the present exampleembodiment.

With reference to FIG. 3 , personal authentication device 210 in thepresent example embodiment includes personal authentication unit 301that verifies an individual from an acoustic characteristic of anacoustic signal propagating through a head of a user. Then, personalauthentication device 210 includes attachment trouble rule storage unit302 that stores an attachment trouble rule, attachment trouble detectionunit 303 that detects a trouble with an attachment state of attachedapparatus 220 when the acoustic characteristic satisfies the attachmenttrouble rule, and attachment trouble notification unit 304 that notifiesthe trouble with the attachment state.

Attachment trouble detection unit 303 includes attachment troubledetection table 303 a, compares an acoustic characteristic calculated inacoustic characteristic calculation unit 313 of personal authenticationunit 301 with a condition of an acoustic characteristic stored inattachment trouble rule storage unit 302, and detects a trouble with anattachment state. Attachment trouble notification unit 304 includesattachment trouble notification message 304 a, and outputs a messagerelated to an attachment trouble.

Personal authentication unit 301 includes acoustic signal transmissionunit 311, acoustic signal observation unit 312, acoustic characteristiccalculation unit 313, feature amount extraction unit 314, useridentification unit 315, and feature amount storage unit 316.

Acoustic signal transmission unit 311 transmits an acoustic signal to apart of a head of a user via an earphone. Herein, the part of the headto which the acoustic signal is transmitted is more specifically aregion of the head in which a cavity is formed, and may be at least apart of a region to which an accessory and an apparatus generating anacoustic effect can be attached or brought close.

Acoustic signal observation unit 312 observes an acoustic signal thathas been transmitted from acoustic signal transmission unit 311 andpropagated through a part of a head of a user. The part of the head as apropagation path of the acoustic signal may be more specifically atleast a part of a skull, a brain, a sensory organ, and a cavitytherebetween, constituting the head.

Acoustic characteristic calculation unit 313 calculates an acousticcharacteristic of an acoustic signal propagating through a part of ahead of a user, based on an acoustic signal transmitted from acousticsignal transmission unit 311 and an acoustic signal observed by acousticsignal observation unit 312. Herein, the acoustic characteristic is,such as, an impulse response, a transfer function acquired by performinga fast Fourier transform on an impulse response.

Feature amount extraction unit 314 calculates a feature amount concernedwith a user from a calculated acoustic characteristic. As a featureamount, for example, a logarithm spectrum, mel-frequency cepstrumcoefficients, linear predictive coding coefficients may be used.

Further, a feature amount acquired by dimensionally compressing alogarithm spectrum, mel-frequency cepstrum coefficients, and linearpredictive coding coefficients by using a principal component analysisor a linear discriminative analysis may be used. Further, a featureamount other than this may be used.

User identification unit 315 compares a feature amount acquired byfeature amount extraction unit 314 with a feature amount of a registereduser stored in feature amount storage unit 316 described later, anddetermines whether or not a user corresponds to a registered individual.

Feature amount storage unit 316 stores a feature amount previouslyextracted from a predetermined user. For example, feature amount storageunit 316 stores a feature amount previously extracted from a pluralityof users by using acoustic signal transmission unit 311, acoustic signalobservation unit 312, acoustic characteristic calculation unit 313, andfeature amount extraction unit 314, or a configuration identical tothese. A verification method in the authentication is to determinewhether or not a person is the same person as a person among people tobe verified having a previously stored feature amount, determine aperson from a plurality of people to be verified having a previouslystored feature amount, or perform these two determinations at the sametime.

(Example of Acoustic Characteristic)

FIGS. 4A and 4B are diagrams illustrating an acoustic characteristicused for personal authentication according to the present exampleembodiment.

FIG. 4A is a diagram illustrating impulse response 410 as an acousticcharacteristic. The impulse response is used for detecting an attachmenttrouble in the present example embodiment.

Further, FIG. 4B is a diagram of logarithm spectrum 420. When anacoustic characteristic of an ear canal is used in the present exampleembodiment, as illustrated in FIG. 4B, a peak is observed for a factorconcerned with volume of a frequency related to a distance (normally 2to 3 cm) between acoustic signal transmission unit 311 and an eardrum.For example, a value at this peak may be compared with a predeterminedreference value in attachment trouble detection in the present exampleembodiment. Further, volume indicates power of a spectrum acquired byusing an amplitude of a speech waveform, a fast Fourier transform (FFT).

(Detection of Attachment Trouble)

For example, when an attachment state is insufficient and a gap ispresent, sound leaks to the outside, and thus volume of a volumecharacteristic becomes lower than that when an attachment state issufficient. By using this change, an insufficient attachment state canbe detected. Further, a change in the volume depends on a shape of anearphone and an attachment state, and characteristically appears at aspecific frequency. Thus, detection focusing on volume at a specificfrequency is effective.

Furthermore, with reference to FIG. 4B, a peak related to a distance of2 to 3 cm to an eardrum occurs at 5 to 8 kHz. This can be understood asair column resonance of a tube of 2 to 3 cm with closed ends. Assumingthat the speed of sound is 340 meters per second, a wavelength andnatural vibration of the tube with closed ends can be expressed by thefollowing expressions.

λn=(2/n)×L n=1,2,3, . . .

fn=(340/2L)×n n=1,2,3, . . .

Herein, λn represents a wavelength (unit is meter), and fn represents afrequency (unit is Hz). n=1 represents 1 harmonic of natural vibration,and n=2, 3, 4 respectively represent 2 harmonics, 3 harmonics, and 4harmonics of natural vibration. L represents a length (unit is meter) ofan air column, and herein represents a distance from a headphone or anearphone to an eardrum.

In this way, it is considered that a frequency of 1 harmonic of naturalvibration could not be observed when a frequency of natural vibrationbecame a constant multiple of equal to or greater than two times apredetermined value, and it can thus be determined that attachment of aheadphone or an earphone is insufficient.

A wavelength and natural vibration of a tube of 2 to 3 cm with one openend are expressed by the following expressions.

λn={4/(2n−1)}×L n=1,2,3, . . .

fn={340/4L}×(2n−1)n=1,2,3, . . .

In this way, it is considered that, when a frequency of naturalvibration halved, natural vibration of a tube with closed ends changedto that of a tube with one open end, and it can thus be determined thatattachment of a headphone or an earphone is insufficient.

(Attachment Trouble Rule-Storage Unit)

FIG. 5A is a diagram illustrating a configuration of attachment troublerule storage unit 302 according to the present example embodiment.Attachment trouble rule storage unit 302 stores an attachment troublerule for detecting an attachment trouble from an acousticcharacteristic. Note that, a configuration of attachment trouble rulestorage unit 302 is not limited to FIG. 5A.

Attachment trouble rule storage unit 302 stores, in association withrule number 511 that identifies an attachment trouble rule, subjectfactor 512 of acoustic characteristic used for attachment troubledetection, attachment trouble condition 513 in subject factor 512 ofacoustic characteristic, and combination algorithm 514 of attachmenttrouble rules used for attachment trouble detection. Note that, eachattachment trouble rule may be used alone.

The following examples of attachment trouble rules in the presentexample embodiment are conceivable. Herein, a factor represents one ormore values among a plurality of values constituting acousticcharacteristics.

-   -   (Rule Number 11): A value of a factor equivalent to volume among        acoustic characteristics is smaller or greater than a        predetermined reference.    -   (Rule Number 12): A value of a factor equivalent to volume at a        low frequency (for example, equal to or less than 1 kHz) among        acoustic characteristics is smaller than (falls below) or        greater than (exceeds) a predetermined reference.    -   (Rule Number 13): A value of a factor equivalent to volume at a        high frequency (for example, equal to or greater than 1 kHz)        among acoustic characteristics is smaller or greater than a        predetermined reference.    -   (Rule Number 14): A value of a factor concerned with volume of        characteristic vibration/natural vibration related to a distance        from an earphone (acoustic transmission unit) to an eardrum        among acoustic characteristics is smaller or greater than a        predetermined reference.    -   (Rule Number 15): A maximum value and a value (peak) greater        than two adjacent values among factors equivalent to volume of        an acoustic characteristic are smaller or greater than a        predetermined reference.    -   (Rule Number 16): A frequency of natural vibration related to a        distance from an earphone (acoustic transmission unit) to an        eardrum among acoustic characteristics is a constant multiple or        half of a predetermined reference.    -   (Rule Number 17): A frequency of a factor having a maximum value        and a factor having a value (peak) greater than two adjacent        values, among factors equivalent to volume of an acoustic        characteristic, is a constant multiple or half of a frequency of        a corresponding factor of an acoustic characteristic acquired        from a predetermined person to be authenticated.    -   (Rule Number 18): One or more factors of an acoustic        characteristic are greatly different from an average of acoustic        characteristics previously acquired from a plurality of people.    -   (Rule Number 19): One or more factors of an acoustic        characteristic are similar to an acoustic characteristic in        which a trouble with attachment is previously found.

(Attachment Trouble Detection Table)

FIG. 5B is a diagram illustrating a configuration of attachment troubledetection table 303 a according to the present example embodiment.

Attachment trouble detection table 303 a is used as a temporary memoryfor detecting an attachment trouble by attachment trouble detection unit303.

Attachment trouble detection table 303 a stores, in association withsubject factor 521 of attachment trouble detection, subject factor 522of acoustic characteristic from acoustic characteristic calculation unit313, attachment trouble rule number 523 used for attachment troubledetection, and attachment trouble determination flag 524 when subjectfactor 522 of acoustic characteristic satisfies a condition ofattachment trouble rule number 523. Then, attachment trouble detectiontable 303 a stores attachment trouble detection output 525 that combinesattachment trouble determination flag 524 according to combinationalgorithm 514 of attachment trouble rule.

<<Hardware Configuration of Personal Authentication Device>>

FIG. 6 is a block diagram illustrating a hardware configuration ofpersonal authentication device 210 according to the present exampleembodiment.

In FIG. 6 , CPU 610 is a processor for arithmetic control and achievesthe functional structural unit in FIG. 3 by executing a program. ROM 620stores fixed data, such as initial data and a program, and a program.Network interface 630 controls communication with an external device viaa network.

RAM 640 is a random access memory used as a work area for a temporarymemory by CPU 610. RAM 640 secures a region in which data needed forachieving the present example embodiment are stored. Transmissionacoustic signal data 641 are data about a transmission acoustic signalto an earphone. Observation acoustic signal data 642 are data about anobservation acoustic signal from a microphone. Acoustic characteristicdata 643 are data about an acoustic characteristic calculated fromtransmission acoustic signal data 641 and observation acoustic signaldata 642 by acoustic characteristic calculation unit 313. Feature amountextraction data 644 are data extracted from acoustic characteristic data643 by feature amount extraction unit 314. User identification result645 is data about a user identified from feature amount extraction data644 and feature amount storage unit 316 by user identification unit 315.Attachment trouble detection table 303 a is a table used for detectingan attachment trouble by attachment trouble detection unit 303, which isdescribed with FIG. 5B. Attachment trouble notification message 304 a isa message with which attachment trouble notification unit 304 notifiesan attachment trouble. Input-output data 646 are data input to andoutput from a peripheral apparatus via input-output interface 660.Transmit-receive data 647 are data communicated via network interface630.

Storage 650 stores a database, various parameters, or the following dataor program needed for achieving the present example embodiment.Attachment trouble rule storage unit 302 is a database of an attachmenttrouble rule, which is described with FIG. 5A. Feature amount storageunit 316 is a database that stores a feature amount previously extractedfrom a predetermined user. Personal authentication algorithm 651 is analgorithm for a personal authentication method in the present exampleembodiment. Attachment trouble determination algorithm 652 is analgorithm for a method of determining an attachment trouble in thepresent example embodiment. Attachment trouble notification message 653is a database that stores a notification message related to anattachment trouble.

The following program is stored in storage 650. Personal authenticationcontrol program 654 is a program for controlling personal authenticationdevice 210. Acoustic signal-transmission observation module 655 is amodule that controls transmission and observation of acoustic signals.Acoustic characteristic calculation module 656 is a module thatcalculates an acoustic characteristic from transmitted and observedacoustic signals. Feature amount extraction module 657 is a module thatextracts a feature amount from an acoustic characteristic. Useridentification module 658 is a module that identifies a user from afeature amount extracted from an acoustic characteristic and a featureamount of feature amount storage unit 316. Attachment troubledetermination module 659 is a module that determines an attachmenttrouble with attached apparatus 220 from an acoustic characteristic (ora feature amount) according to an attachment trouble rule.

Input-output interface 660 interfaces input and output with a peripheralapparatus. Display unit 661, operation unit 662, voice input-output unit663, and earphone/microphone control unit 664 that connects attachedapparatus 220 are connected to input-output interface 660.

Note that, a program and data related to a general-purpose function andother achievable function of personal authentication device 210 are notillustrated in RAM 640 and storage 650 in FIG. 6 .

<<Processing Procedure of Personal Authentication Device>>

FIG. 7 is a flowchart illustrating a processing procedure of personalauthentication device 210 according to the present example embodiment.This flowchart is executed by CPU 610 in FIG. 6 using RAM 640, andachieves the functional structural unit in FIG. 3 .

In Step S701, personal authentication device 210 transmits an acousticsignal to a part of a head of a user. In Step S703, personalauthentication device 210 observes an acoustic signal propagatingthrough the part of the head. In Step S705, personal authenticationdevice 210 calculates an acoustic characteristic from the transmittedacoustic signal and the observed acoustic signal.

In Step S707, personal authentication device 210 executes attachmenttrouble determination processing by using the calculated acousticcharacteristic. In Step S709, personal authentication device 210determines whether an attachment trouble flag is ON or OFF. When theattachment trouble flag is ON, personal authentication device 210notifies an attachment trouble in Step S715, the processing returns toStep S701, and personal authentication is repeated. When the attachmenttrouble flag is OFF, personal authentication device 210 extracts afeature amount from the acoustic characteristic in Step S711. Then, inStep S713, personal authentication device 210 compares the featureamount with a previously stored feature amount and authenticates theuser.

(Attachment Trouble Determination Processing)

FIG. 8 is a flowchart illustrating a procedure for the attachmenttrouble determination processing (S707) according to the present exampleembodiment.

In Step S801, personal authentication device 210 acquires an attachmenttrouble rule from attachment trouble rule storage unit 302. In StepS803, personal authentication device 210 determines whether or not it isan attachment trouble from a calculated acoustic characteristicaccording to the attachment trouble rule.

When it is not an attachment trouble, personal authentication device 210sets an attachment trouble flag to OFF in Step S805, and the processingreturns. On the other hand, when it is an attachment trouble, personalauthentication device 210 sets the attachment trouble flag to ON in StepS807, and the processing returns.

According to the present example embodiment, when personalauthentication device 210 measures an acoustic characteristic of an earcanal of a user by using a headphone or an earphone including amicrophone embedded therein, personal authentication device 210 detectswhether or not an attachment condition of the headphone or the earphoneis suitable for personal authentication. When the attachment conditionis not suitable for the personal authentication, personal authenticationdevice 210 prevents a false determination due to the attachmentcondition of the headphone or the earphone by giving feedback to theuser. In this way, a false determination due to an attachment conditionof an apparatus that transmits and receives an acoustic signal can beprevented, and accurate personal authentication can be performed.

In other words, a user to be authenticated wears a headphone or anearphone including a microphone embedded therein, and thus an acousticcharacteristic of an acoustic signal propagating through a head of theuser is measured, and personal authentication is performed. At thistime, personal authentication device 210 detects that an attachmentstate of the attached headphone or earphone is not an appropriate stateby comparing the acoustic characteristic with an attachment troublerule. In this way, personal authentication device 210 can prevent twofalse determinations caused by an attachment trouble that are falserejection in which the person is mistakenly rejected and falseacceptance in which another person is mistakenly accepted. Thus,personal authentication device 210 can achieve the effect of the presentexample embodiment.

Third Example Embodiment

Next, a personal authentication device according to a third exampleembodiment of the present invention is described. The personalauthentication device according to the present example embodiment isdifferent from that in the second example embodiment described above inthat an attachment trouble is detected by using an acousticcharacteristic acquired from a specific authentication subject forcomparison instead of using a preset reference. In other words, thepersonal authentication device according to the present exampleembodiment stores an acoustic characteristic acquired from an acousticsignal previously acquired from a person to be authenticated. Thepersonal authentication device according to the present exampleembodiment determines that a trouble has occurred in an attachment stateof an apparatus when a relationship between a calculated acousticcharacteristic and a previously stored acoustic characteristic satisfiesan attachment trouble rule. The other configuration and operation aresimilar to those in the second example embodiment, and thus the sameconfiguration and operation are provided with the same reference signs,and detailed description thereof is omitted.

<<Functional Configuration of Personal Authentication Device>>

FIG. 9 is a block diagram illustrating a functional configuration ofpersonal authentication device 910 according to the present exampleembodiment. Note that, a functional structural unit in FIG. 9 similar tothat in FIG. 3 is provided with same reference sign, and the descriptionthereof is not repeated.

Attachment trouble rule storage unit 902 stores an attachment troublerule used in the present example embodiment. Acoustic characteristicstorage unit 905 stores an acoustic characteristic acquired from anacoustic signal previously acquired from a person to be authenticated.Attachment trouble detection unit 903 includes attachment troubledetection table 903 a, and detects an attachment trouble from anacoustic characteristic calculated by acoustic characteristiccalculation unit 313 and an acoustic characteristic stored in acousticcharacteristic storage unit 905 according to an attachment trouble rule.

(Attachment Trouble Rule-Storage Unit)

FIG. 10A is a diagram illustrating a configuration of attachment troublerule storage unit 902 according to the present example embodiment.Attachment trouble rule storage unit 902 stores an attachment troublerule for detecting an attachment trouble from an acousticcharacteristic. Note that, a configuration of attachment trouble rulestorage unit 902 is not limited to FIG. 10A.

Attachment trouble rule storage unit 902 stores, in association withrule number 1011 that identifies an attachment trouble rule, subjectfactor 1012 of acoustic characteristic used for attachment troubledetection, attachment trouble condition 1013 in subject factor 1012 ofacoustic characteristic, and combination algorithm 1014 of attachmenttrouble rules used for attachment trouble detection. Note that, eachattachment trouble rule may be used alone.

The following examples of attachment trouble rules in the presentexample embodiment are conceivable.

-   -   (Rule Number 21): A value of a factor equivalent to volume among        acoustic characteristics is smaller or greater than a value of a        factor equivalent to an acoustic characteristic previously        acquired from a person to be authenticated.    -   (Rule Number 22): A value of a factor equivalent to a low        frequency (for example, equal to or less than 1 kHz) among        acoustic characteristics is smaller or greater than a value of a        factor equivalent to an acoustic characteristic previously        acquired from a person to be authenticated.    -   (Rule Number 23): A value of a factor equivalent to volume at a        high frequency (for example, equal to or greater than 1 kHz)        among acoustic characteristics is smaller or greater than a        value of a factor equivalent to an acoustic characteristic        previously acquired from a person to be authenticated.    -   (Rule Number 24): A factor concerned with volume of natural        vibration related to a distance from an earphone (acoustic        transmission unit) to an eardrum among acoustic characteristics        is smaller or greater than a value of a factor equivalent to an        acoustic characteristic previously acquired from a person to be        authenticated.    -   (Rule Number 25): A maximum value and a value (peak) greater        than two adjacent values among factors equivalent to volume of        an acoustic characteristic are smaller than or greater than a        value of a corresponding factor of an acoustic characteristic        previously acquired from a person to be authenticated.    -   (Rule Number 26): A frequency of natural vibration related to a        distance from an earphone (acoustic transmission unit) to an        eardrum among acoustic characteristics is a constant multiple or        half of a frequency of corresponding natural vibration of an        acoustic characteristic previously acquired from a person to be        authenticated.    -   (Rule Number 27): A frequency of a factor with a maximum value        and a fator with a value (peak) greater than two adjacent        values, among factors equivalent to volume of an acoustic        characteristic, is a constant multiple or half of a frequency of        a corresponding factor of an acoustic characteristic previously        acquired from a person to be authenticated.

(Acoustic Characteristic Storage Unit)

FIG. 10B is a diagram illustrating a configuration of acousticcharacteristic storage unit 905 according to the present exampleembodiment. Note that, a configuration of acoustic characteristicstorage unit 905 is not limited to FIG. 10B.

Acoustic characteristic storage unit 905 stores, in association withperson to be authenticated 1051, one or a plurality of attachmenttrouble rule numbers 1052 and previously acquired information 1053 abouta subject factor of an acoustic characteristic corresponding toattachment trouble rule number 1052.

(Attachment Trouble Detection Table)

FIG. 10C is a diagram illustrating a configuration of attachment troubledetection table 903 a according to the present example embodiment.Attachment trouble detection table 903 a is used as a temporary memoryfor detecting an attachment trouble by attachment trouble detection unit903. Note that, a structural component in FIG. 10C similar to that inFIG. 5B is provided with same reference sign, and the descriptionthereof is not repeated.

Subject factor 1022 of acoustic characteristic is information acquiredin association with attachment trouble rule number 1023 from acousticcharacteristic storage unit 905. Attachment trouble rule number 1023represents an attachment trouble rule used for determining an attachmenttrouble.

(Attachment Trouble Determination Processing)

FIG. 11 is a flowchart illustrating a procedure for attachment troubledetermination processing (S707) according to the present exampleembodiment. Note that, the processing procedure in personalauthentication device 910 is similar to that in FIG. 7 , and FIG. 11 mayreplace Step S707 in FIG. 7 . Further, a step in FIG. 11 similar to thatin FIG. 8 is provided with same step number, and the description thereofis not repeated.

In Step S1102, personal authentication device 910 acquires an acousticcharacteristic related to an attachment trouble rule from acousticcharacteristic storage unit 905.

According to the present example embodiment, by setting a differentattachment trouble rule for each person to be authenticated, a falsedetermination due to an attachment condition of an apparatus thattransmits and receives an acoustic signal can be prevented, and accuratepersonal authentication can be performed.

Fourth Example Embodiment

Next, a personal authentication device according to a fourth exampleembodiment of the present invention is described. The personalauthentication device according to the present example embodiment isdifferent from that in the second example embodiment and the thirdexample embodiment described above in that an attachment trouble isdetected, based on a feature amount extracted from an acousticcharacteristic. Note that, the personal authentication device may use afeature amount extracted in the present example embodiment and anacoustic characteristic in the second example embodiment separately orin combination for detecting an attachment trouble. Further, thepersonal authentication device may be configured to detect an attachmenttrouble by comparing a feature amount extracted in the present exampleembodiment with a feature amount stored in a feature amount storage unitof a personal authentication unit. The other configuration and operationare similar to those in the second example embodiment, and thus the sameconfiguration and operation are provided with the same reference signs,and detailed description thereof is omitted.

<<Functional Configuration of Personal Authentication Device>>

FIG. 12 is a block diagram illustrating a functional configuration ofpersonal authentication device 1210 according to the present exampleembodiment. Note that, a functional structural unit in FIG. 12 similarto that in FIG. 3 is provided with same reference sign, and thedescription thereof is not repeated.

Attachment trouble rule storage unit 1202 stores an attachment troublerule used in the present example embodiment. Attachment troubledetection unit 1203 includes attachment trouble detection table 1203 a,and detects an attachment trouble when a feature amount extracted byfeature amount extraction unit 314 satisfies a condition stored inattachment trouble rule storage unit 1202.

(Feature Amount)

As a feature amount, for example, a logarithm spectrum, mel-frequencycepstrum coefficients (MFCC), linear predictive coding coefficients maybe used in the present example embodiment. Further, a feature amountacquired by dimensionally compressing a logarithm spectrum,mel-frequency cepstrum coefficients, and linear predictive codingcoefficients by using a principal component analysis or a lineardiscriminative analysis may be used. Further, a feature amount otherthan this may be used.

FIG. 13 is a diagram illustrating related waveform 1300 of MFCC as afeature amount used for personal authentication according to the presentexample embodiment.

(Attachment Trouble Rule-Storage Unit)

FIG. 14A is a diagram illustrating a configuration of attachment troublerule storage unit 1202 according to the present example embodiment.Attachment trouble rule storage unit 1202 stores an attachment troublerule for detecting an attachment trouble from an acousticcharacteristic. Note that, a configuration of attachment trouble rulestorage unit 1202 is not limited to FIG. 14A.

Attachment trouble rule storage unit 1202 stores, in association withrule number 1411 that identifies an attachment trouble rule, subjectfactor 1412 of feature amount used for attachment trouble detection,attachment trouble condition 1413 in subject factor 1412 of featureamount, and combination algorithm 1414 of attachment trouble rules usedfor attachment trouble detection. Note that, each attachment troublerule may be used alone.

Note that, a rule similar to that in the second example embodiment orthe third example embodiment may be used as an attachment trouble rule.Further, when a factor constituting a feature amount is not directlyrelated to volume or volume at a specific frequency, the feature amountmay be restored to an acoustic characteristic by being subjected to anarithmetic computation, which is the inverse of calculation thatcalculates the feature amount, and the acoustic characteristic may beused.

(Attachment Trouble Detection Table)

FIG. 14B is a diagram illustrating a configuration of an attachmenttrouble detection table according to the present example embodiment.Attachment trouble detection table 1203 a is used as a temporary memoryfor detecting an attachment trouble by attachment trouble detection unit1203. Note that, a structural component in FIG. 14B similar to that inFIG. 5B is provided with same reference sign, and the descriptionthereof is not repeated.

Subject factor 1421 of attachment trouble detection represents a subjectfactor of a feature amount used in attachment trouble detection. Subjectfactor 1422 of feature amount is information acquired from featureamount extraction unit 314. Attachment trouble rule number 1423represents an attachment trouble rule in the present example embodimentused for determining an attachment trouble.

<<Processing Procedure of Personal Authentication Device>>

FIG. 15 is a flowchart illustrating a processing procedure of personalauthentication device 1210 according to the present example embodiment.Note that, a step in FIG. 15 similar to that in FIG. 7 is provided withsame step number, and the description thereof is not repeated.

In Step S1507, personal authentication device 1210 executes attachmenttrouble determination processing, based on an attachment trouble rule.Then, in Step S1509, personal authentication device 1210 determineswhether an attachment trouble flag is ON or OFF.

Note that, the attachment trouble determination processing in Step S1507is the same processing in Step S707 in which an acoustic characteristicis replaced by a feature amount.

According to the present example embodiment, by using a feature amountextracted from an acoustic characteristic, a false determination due toan attachment condition of an apparatus that transmits and receives anacoustic signal can be prevented, and accurate personal authenticationcan be performed.

In other words, a user to be authenticated wears a headphone or anearphone including a microphone embedded therein, and thus an acousticcharacteristic of an acoustic signal propagating through a head of theuser is measured, and personal authentication is performed. At thistime, personal authentication device 1210 detects that an attachmentstate of the attached headphone or earphone is not an appropriate stateby comparing the feature amount extracted from the acousticcharacteristic with an attachment trouble rule. In this way, personalauthentication device 1210 can prevent two false determinations causedby an attachment trouble that are false rejection in which the person ismistakenly rejected and false acceptance in which another person ismistakenly accepted.

Thus, personal authentication device 1210 can improve authenticationaccuracy, improve resistance to noise, and reduce an amount of data bydetermining an attachment trouble, based on a feature amount.

Fifth Example Embodiment

Next, a personal authentication device according to a fifth exampleembodiment of the present invention is described. The personalauthentication device according to the present example embodiment isdifferent from that in the second example embodiment to the fourthexample embodiment described above in that an acoustic characteristic iscalculated from an acoustic signal propagating through a bone of a headinstead of an acoustic signal of an ear canal, and a feature amount isextracted. While sound generally propagates through mainly air as amedium, sound can also propagate through a bone structure as a medium.Propagation of sound through a bone as a medium is referred to as boneconduction. A propagation characteristic of bone conduction also hasindividuality, and can thus be used for authentication of an individual.Note that, the present example embodiment may also adopt a configurationof attachment trouble detection using a feature amount as in the fourthexample embodiment and a configuration in which an acousticcharacteristic and a feature amount of an authentication subject areused for comparison as in the third example embodiment. The otherconfiguration and operation are similar to those in the second exampleembodiment to the fourth example embodiment, and thus the sameconfiguration and operation are provided with the same reference signs,and detailed description thereof is omitted.

<<Functional Configuration of Personal Authentication Device>>

FIG. 16 is a block diagram illustrating a functional configuration ofpersonal authentication device 1610 according to the present exampleembodiment. Note that, a functional structural unit in FIG. 16 similarto that in FIG. 3 is provided with same reference sign, and thedescription thereof is not omitted.

Bone conduction signal-transmission unit 1611 transmits a boneconduction signal being an acoustic signal for bone conduction to a partof a head. Herein, the part of the head to which the bone conductionsignal is transmitted is more specifically a region of the head in whicha bone is formed, and may be at least a part of a region to which anaccessory and an apparatus generating an acoustic effect can be attachedor brought close.

Bone conduction signal-observation unit 1612 observes a bone conductionsignal (acoustic signal) that has been transmitted from bone conductionsignal-transmission unit 1611 and propagated through the part of thehead of a user. Herein, the part of the head as a propagation path ofthe bone conduction signal being the acoustic signal for bone conductionmay be more specifically at least a part of a skull constituting a head,a tooth, a brain, a sensory organ, and a cavity between them. Note that,it is assumed that the propagation path includes at least a bone.

Bone conduction signal-observation unit 1612 may be achieved by, forexample, a bone conduction microphone. At this time, bone conductionsignal-observation unit 1612 may observe an acoustic signal from a partdifferent from an arbitrary part of a head to which bone conductionsignal-transmission unit 1611 transmits an acoustic signal.

According to the present example embodiment, by using bone conduction ofan acoustic signal in a head, a false determination due to an attachmentcondition of an apparatus that transmits and receives an acoustic signalcan be prevented, and accurate personal authentication can be performed.

Other Example Embodiment

While personal authentication processing and attachment troubledetection processing are performed by a PC in the example embodimentsdescribed above, a personal authentication device, a personalauthentication method, and a personal authentication program thatauthenticate an individual by using an audio device can also beprovided.

Although the claimed invention has been described with reference to theexample embodiments, it should be understood that the claimed inventionis not limited to the above-described example embodiments. Variousmodifications that can be understood by those skilled in the art withinthe scope of the claimed invention may be applied to the configurationand the details of the claimed invention. Further, a system or a devicethat combines different features included in the respective exampleembodiments in any way is also included within the scope of the presentinvention.

Further, the present invention may be applied to a system including aplurality of apparatuses and may be applied to a single device.Furthermore, the present invention is also applicable to a case where aninformation processing program realizing functions of the exampleembodiments is supplied to a system or a device directly or remotely.Therefore, in order to realize functions of the present invention by acomputer, a program installed in the computer or a medium that storesthe program, and a World Wide Web (WWW) server that causes the programto be downloaded are also included within the scope of the presentinvention. In particular, a non-transitory computer readable medium thatstores a program causing a computer to execute a processing stepincluded in the above-described example embodiments is included withinthe scope of the present invention.

Other Expression of Example Embodiment

A part or the whole of the above-described example embodiments may alsobe described in supplementary notes below, which is not limited thereto.

The present invention has been described above by taking theabove-described example embodiments as exemplary examples. However, thepresent invention is not limited to the above-described exampleembodiments. In other words, various aspects apparent to those skilledin the art may be applied to the present invention within the scope ofthe present invention.

REFERENCE SIGNS LIST

-   -   100 Personal authentication device    -   101 Personal authentication unit    -   102 Attachment trouble rule storage unit    -   200 Personal authentication system    -   210 Personal authentication device    -   220 Attached apparatus    -   230 Sound processor    -   240 Microphone amplifier    -   301 Personal authentication unit    -   302 Attachment trouble rule storage unit    -   303 Attachment trouble detection unit    -   304 Attachment trouble notification unit    -   610 CPU    -   620 ROM    -   630 Network interface    -   640 RAM    -   650 Storage    -   660 Input-output interface    -   661 Display unit    -   662 Operation unit    -   663 Voice input-output unit    -   664 Earphone/microphone control unit    -   902 Attachment trouble rule storage unit    -   903 Attachment trouble detection unit    -   905 Acoustic characteristic storage unit    -   910 Personal authentication device    -   1202 Attachment trouble rule storage unit    -   1203 Attachment trouble detection unit    -   1210 Personal authentication device    -   1610 Personal authentication device

1. A personal authentication device authenticating a user, the personalauthentication device comprises; a memory configured to storeinstructions; and at least one processor configured to execute theinstructions to perform: acquiring acoustic characteristic of the userfrom an acoustic signal transferred a head of the user; determiningwhether the acoustic characteristic of the user or a feature valueextracted from the acoustic characteristic satisfies a predeterminedcondition in accordance with an attachment trouble rule; and detectingthat there is an attachment trouble how the user wears the personalauthentication device based on the acoustic characteristic of the useror the feature value extracted from the acoustic characteristic, whereinthe at least one processor configured to execute the instructions toperform: when detecting that there is an attachment trouble how the userwears the personal authentication device, giving feedback to the userthat it is not suitable for the person authentication.
 2. The personalauthentication device according to claim 1, wherein the acousticcharacteristic includes an impulse response, or a transfer functionacquired by performing a Fourier transform or a Laplace transform on theimpulse response.
 3. The personal authentication device according toclaim 1, wherein the attachment trouble rule is described on anattachment trouble detection table which associates an attachmenttrouble rule number and an attachment trouble determination flag with anattachment trouble detection subject factor and a feature value subjectfactor.
 4. A personal authentication method for authenticating a user,the personal authentication method comprises; acquiring acousticcharacteristic of the user from an acoustic signal transferred a head ofthe user; determining whether the acoustic characteristic of the user ora feature value extracted from the acoustic characteristic satisfies apredetermined condition in accordance with an attachment trouble rule;and detecting that there is an attachment trouble how the user wears thepersonal authentication device based on the acoustic characteristic ofthe user or the feature value extracted from the acousticcharacteristic, wherein the personal authentication method comprises:when detecting that there is an attachment trouble how the user wearsthe personal authentication device, giving feedback to the user that itis not suitable for the person authentication.
 5. The personalauthentication method according to claim 4, wherein the acousticcharacteristic includes an impulse response, or a transfer functionacquired by performing a Fourier transform or a Laplace transform on theimpulse response.
 6. The personal authentication method according toclaim 4, wherein the attachment trouble rule is described on anattachment trouble detection table which associates an attachmenttrouble rule number and an attachment trouble determination flag with anattachment trouble detection subject factor and a feature value subjectfactor.
 7. A non-transitory recording medium having a personauthentication program stored thereon, the person authentication programcauses a computer to perform: acquiring acoustic characteristic of theuser from an acoustic signal transferred a head of the user; determiningwhether the acoustic characteristic of the user or a feature valueextracted from the acoustic characteristic satisfies a predeterminedcondition in accordance with an attachment trouble rule; and detectingthat there is an attachment trouble how the user wears the personalauthentication device based on the acoustic characteristic of the useror the feature value extracted from the acoustic characteristic, whereinthe person authentication program causes the computer to perform: whendetecting that there is an attachment trouble how the user wears thepersonal authentication device, giving feedback to the user that it isnot suitable for the person authentication.
 8. The non-transitoryrecording medium according to claim 7, wherein the acousticcharacteristic includes an impulse response, or a transfer functionacquired by performing a Fourier transform or a Laplace transform on theimpulse response.
 9. The non-transitory recording medium according toclaim 7, wherein the attachment trouble rule is described on anattachment trouble detection table which associates an attachmenttrouble rule number and an attachment trouble determination flag with anattachment trouble detection subject factor and a feature value subjectfactor.